feat(audit): a run manifest should answer questions about its own run - #55
Merged
Conversation
Review finding #1. ScenarioManifest recorded `vendor` and `duration`; RunManifest recorded neither, and nothing recorded the rate actually in force or what the socket really did. A manifest that cannot say which profile rendered it is a weaker audit record than safety rule 5 implies, and the web layer had to paper over the gap. Four fields, all defaulted so manifests written before them still load, which is the convention `pace` and `speed` already set: - `vendor` which dialect rendered these events - `duration` the --duration asked for, verbatim - `rate` the events-per-second ceiling in force - `send_stats` sends, bytes, errors, oversize `send_stats` is None rather than a row of zeroes when a run had no collector, because zeroes read like a send that delivered nothing. `event_count` counts events rendered; `send_stats["sends"]` counts datagrams the kernel accepted, and the two differing is the interesting case rather than an inconsistency. Captured on every exit path, so a run that failed part-way still records what its socket managed before it died. Read with getattr rather than `emitter.stats`, because the emitter is an injection point and several test doubles implement only send/close. Requiring the attribute would make this record cost every future fake a field it does not otherwise need. Four existing test doubles proved the point immediately. 875 py. black, ruff, mypy clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Review finding #1. ScenarioManifest recorded
vendorandduration;RunManifest recorded neither, and nothing recorded the rate actually in force or
what the socket really did. A manifest that cannot say which profile rendered it
is a weaker audit record than safety rule 5 implies, and the web layer had to
paper over the gap.
Four fields, all defaulted so manifests written before them still load, which is
the convention
paceandspeedalready set:vendorwhich dialect rendered these eventsdurationthe --duration asked for, verbatimratethe events-per-second ceiling in forcesend_statssends, bytes, errors, oversizesend_statsis None rather than a row of zeroes when a run had no collector,because zeroes read like a send that delivered nothing.
event_countcountsevents rendered;
send_stats["sends"]counts datagrams the kernel accepted, andthe two differing is the interesting case rather than an inconsistency.
Captured on every exit path, so a run that failed part-way still records what its
socket managed before it died.
Read with getattr rather than
emitter.stats, because the emitter is aninjection point and several test doubles implement only send/close. Requiring the
attribute would make this record cost every future fake a field it does not
otherwise need. Four existing test doubles proved the point immediately.
875 py. black, ruff, mypy clean.